Generalized reinforcement learning in perfect-information games

This paper studies action-based reinforcement learning in finite perfectinformation games. Restrictions on the valuation updating rule that that are necessary and sufficient for the play to converge to a subgame-perfect Nash equilibrium (SPNE) are identified. These conditions encompass well-known examples of reinforcement learning and are mild enough to contain other interesting and plausible learning behavior. We provide examples of such updating rule that suggest that the extent of knowledge and rationality assumptions needed to support a SPNE outcome in finite perfect-information games may be minimal.